1. Prerequisites

None.

2. Introduction

In case such as the Visium platform, spatial transcriptomic output comes with a histology image. SPATA2 offers many functions for histology specific analysis. Therefore, detailed interaction between image dimensions and extent and barcode-spot positions in form of data coordinates is required.

This vignette is split in two parts. The first one introduces the basic functions that ensure alignment between image and barcode-spot coordinates and that allow to exchange low- and high resolution image that come with any 10X Visium output. The second part goes beyond what is necessary to deal with the 10X Visium output and explains in depth how images are stored and dealt with.

Throughout this vignette we work with the glioblastoma sample UKF313T. You can download the raw output folder here.

library(SPATA2)
library(SPATAData)
library(tidyverse)

object_t313 <- downloadFromPublication(pub = "kueckelhaus_et_al_2024", id = "UKF313T")

object_t313 <- setDefault(object_t313, display_image = TRUE, pt_size = 1.5)

3. Basic image handling

The following exemplifies how to use basic image related functions.

3.1 Extracting image data

To obtain the image, that is currently set as well as meta information there exist several functions prefixed with getImage*(). To name a few:


# the image 
getImage(object_t313)
## Image 
##   colorMode    : Color 
##   storage.mode : double 
##   dim          : 592 600 3 
##   frames.total : 3 
##   frames.render: 1 
## 
## imageData(object)[1:5,1:6,1]
##      [,1] [,2] [,3] [,4] [,5] [,6]
## [1,]    1    1    1    1    1    1
## [2,]    1    1    1    1    1    1
## [3,]    1    1    1    1    1    1
## [4,]    1    1    1    1    1    1
## [5,]    1    1    1    1    1    1

# image dimensions in width, height and colors
getImageDims(object_t313)
## [1] 592 600   3

# image range in terms of data coordinates
getImageRange(object_t313)
## $x
## [1]   1 592
## 
## $y
## [1]   1 600

3.2 Plotting images

The function plotImage() visualizes the image. The arguments xrange and yrange can be used to zoom in on specific regions.

plotImage(object_t313) + 
  ggpLayerRect(object_t313, xrange = c(200, 400), yrange = c(200, 400))

plotImage(object_t313, xrange = c(200, 400), yrange = c(200, 400))
Fig.1 Visualization of the image.Fig.1 Visualization of the image.

Fig.1 Visualization of the image.

As detailed below, SPATA2 allows to store multiple images. The names of the images currently registered in the SPATA2 object can be obtained by getImageNames().

# right now, this object knows only one image 
getImageNames(object_t313)
## [1] "lowres"

4. Registering new images

To integrate additional images in the workflow they must be registered for the SPATA2 object to know them. Registering a new image means, that a new container for the respective image is created. (For more details on these containers read the documentation of the classes ?SpatialData and ?HistoImage).

As a first example, we register the high resolution image which the Visium platform always provides, usually named somewhat like spatial/tissue_hires_image.png. If you have not already registered the high resolution image with initiateSpataObjectVisium() you can do that with registerImage().

object_t313 <- 
  registerImage(
    object = object_t313, 
    img_name = "hires", # the future name of the image
    dir = "data/outs/spatial/tissue_hires_image.png" # adjust to your directory
  )

# a second image has been registered
getImageNames(object_t313)
## [1] "lowres" "hires"

Generally speaking, registering an image means to set up the container for the image and deposit a directory from where to read it. We recommend to register images with their file directory, since storing multiple images in the SPATA2 object lets the object grow in storage size quickly. This way, the file directory is used to load the image everytime the image is required.

# the hires image alone needs almost 100mb 
getImage(object_t313, img_name = "hires") %>% 
  object.size()
## 94752840 bytes

Note, that in the previous code chunk the argument img_name was specified to obtain the image. In the code chunk where we exemplified plotImage() the image name was not specified. This is because the SPATA2 object only knew one image at that time. Now that it knows two images a decision must be made, which image to use whenever an image is required. In that case the SPATA2 object defaults to the active image.

# check which image is currently active
activeImage(object_t313)
## [1] "lowres"

To change the active image use activateImage().

object_t313 <- activateImage(object_t313, img_name = "hires")

# the default image to use has been switched to 'hires'
activeImage(object_t313)
## [1] "hires"
# ... it is chosen by default 
getImageDims(object_t313)
## [1] 1974 2000    3

By default, the SPATA2 image loads the active image, meaning that the data actually exists in the object. Therefore, the active image must not be read using the file directory with which the image was registered. This accelerates image handling with the image that is currently of interest, the active image. If you have enough RAM, you can always load the image data of inactive images using loadImage() to accelerate access to them when referring to them within functions.

object.size(object_t313)
## 567876152 bytes
object_t313 <- loadImage(object_t313, img_name = "lowres")

# the object grows
object.size(object_t313)
## 576400952 bytes
# the active image must not be specified (regardless whether the image data is loaded or not)
plotImage(object_t313)

# every other image must be specified
plotImage(object_t313, img_name = "lowres")

object_t313 <- unloadImage(object_t313, img_name = "lowres")

object.size(object_t313)
## 567876152 bytes
Fig.2 High- and low resolution image.Fig.2 High- and low resolution image.

Fig.2 High- and low resolution image.

5. Processing images

SPATA2 offers some image processing functions that allow deeper interaction with the histological tissue slide. The function identifyPixelContent() aims to identify the main tissue slide(s), potential tissue fragments as well as artefacts.

object_t313 <- activateImage(object_t313, img_name = "lowres")

object_t313 <- identifyPixelContent(object_t313)

plotImage(object_t313)
plotPixelContent(object_t313)
object_t313 <- activateImage(object_t313, img_name = "lowres")

plotImage(object_t313)
plotPixelContent(object_t313)
Visualization of the identified pixel content.Visualization of the identified pixel content.

Visualization of the identified pixel content.

The content can be used to identify the tissue outline based on the image if method = 'image'. Note that method = 'obs' identifies the tissue outline based on the observations. The vignettes on how to initiate and process SPATA2 objects depending on the platform elaborate on this option. Both outlines can be identified and stored simultaneously in the SPATA2 object.

object_t313 <- identifyTissueOutline(object_t313, method = "image")
plotImage(object_t313, outline = T, fragments = "red")

While the tissue outline based on histology can be used for spatial outlier detection, too, it is particularly useful for image alignment.

6. Aligning images

If you want to integrate images from other sources, they might need to be aligned with the spatial data of the SPATA2 object. Click here to download an example image that is not aligned with the spatial data of the object used in this vignette.

Here, we must introduce the concept of the reference image with which we refer to the image that is aligned with the observations of the SPATA2 object (here, barcoded spots). By default, it is the first image you register or initiate the SPATA2 object with.

refImage(object_t313)
## [1] "lowres"
object_t313 <- 
  registerImage(
    object = object_t313, 
    img_name = "badly_aligned",
    dir = "data/outs/spatial/tissue_image_badly_aligned.png"# adjust to your directory
  )

plotImage(object_t313, img_name = "badly_aligned")

object_t313 <- activateImage(object_t313, img_name = "badly_aligned") 

plotSurface(object = object_t313, display_image = T) + 
  ggpLayerFrameByImage(object_t313)
Fig.4 The reference image and a badly aligned image.Fig.4 The reference image and a badly aligned image.

Fig.4 The reference image and a badly aligned image.

To manipulate the justification of every single image registered in the SPATA2 object you can make use of multiple functions. Each of them do the following: They change the instructions with which the image is transformed upon extraction whenever it is used.

# extract the transformation instructions for an image
getImageTransformations(object_t313, img_name = "lowres")
## $angle
## [1] 0
## 
## $flip
## $flip$horizontal
## [1] FALSE
## 
## $flip$vertical
## [1] FALSE
## 
## 
## $stretch
## $stretch$horizontal
## [1] 1
## 
## $stretch$vertical
## [1] 1
## 
## 
## $translate
## $translate$horizontal
## [1] 0
## 
## $translate$vertical
## [1] 0

… to be continued …